from IPython.display import Image
Image(filename='../imgs/banner.png')

%load_ext pretty_jupyter

Authors:

• Bailey Passmore, Data Scientist, HRDAG
• Larry Barrett, Consultant, HRDAG

import pendulum

Wednesday, 05 March 2025 at 09:34 PM (PST)

%%html

<style>
    #Styling {
        font-weight: bold;
        font-family: Helvetica;
    }
</style>

Goal¶

• What we have in dev-pre-restructure is working fine but we're on a short deadline and need to streamline the data. Let's make a barebones table of the core data we need for analysis, including:
  • event_no
  • date_occurred (w/ year_occurred)
  • source_type (a field derived from the initial event type in the data, either 911 call reporting gunfire or Shotspotter GDT alert)
  • event_location (service address if specified)
  • date_dispatched (is this the same as date of arrival?)

Research questions we're working towards¶

1. Is dispatch reported at the same rate for all districts?
2. RE: Soundthinking / Brookings Institute claim that some 80% of gunfire events do not get reported by citizens - is that True in Chicago?
   • When SST alerts aren't matched to 911 calls, what is the typical disposition of such an alert?
   • When 911 calls aren't matched to SST alert, what is the typical disposition?

Time period covered¶

• Earliest date occurred included: '2021-01-01'
• Last date occurred included: '2024-11-05' (when the SST technology was reportedly deactivated; CPD and OEMC_sst only cover to this date)
  • Does it make sense to extend to end of 2024 anyways? the Dec 2024 911 calls won't have a SST match and could be ignored or addressed differently depending on how we handle analysis

Identifying Event type (based on the _inittype field)¶

shotspotter = ['SST', 'PSST', 'MSST'] # keywords provided by CPD in Info sheet
citizencalls = ['SHOTS', 'SHOTSF', 'PERSHO',] # Note: 'PERGUN', 'PERDOW','PERHLP', 'DOMBAT', etc. excluded

Note about what humans can do that SST can't¶

In the OEMC data, in addition to reports originating as calls about shots fired ('SHOTSF') or persons being shot ('PERSHO'), we also see events with an initial label 'PERGUN' and final label referring to shots fired or someone shot. This tells us that not only do Chicagoans report gunfire in general, they may also report early warning signs of conflict involving firearms before any shots occur, giving first responders a head start to arrive on scene and provide potentially life-saving care.

# dependencies
import re
import numpy as np
from datetime import timedelta
import pandas as pd
from geopy.geocoders import Nominatim
from geopy.distance import geodesic

# support methods
def format_count(v):
    return "{:,}".format(v)


def format_prop(prop, decn=1, asperc=True):
    if asperc: prop = prop*100
    return "{}%".format(round(prop, decn))


def report_fields(df, idcol, cols, fillna=False, headn=10):
    data = df[[idcol] + cols].drop_duplicates()
    if fillna:
        count = data[cols].fillna('None reported').value_counts().to_frame().reset_index()
        perc = data[cols].fillna('None reported').value_counts(normalize=True).to_frame().reset_index(
            ).rename(columns={'proportion': 'percent'})
    else:
        count = data[cols].value_counts().to_frame().reset_index()
        perc = data[cols].value_counts(normalize=True).to_frame().reset_index(
            ).rename(columns={'proportion': 'percent'})
    count['count'] = count['count'].apply(format_count)
    perc.percent = perc.percent.apply(format_prop)
    out = pd.merge(count, perc, on=cols)
    return out.head(headn)

# main
colorder = [
    'event_no',
    'date_occurred',
    'date_dispatched',
    'location',
    'location_x',
    'location_y',
    'init_type',
    'init_type_desc',
    'fin_type',
    'fin_type_desc',
    'disposition',
    'source_type',
    'early_warning',
]
data = pd.read_parquet("../../merge/output/subset.parquet").rename(columns={
    'event_type': 'source_type',
    'event_type_init': 'init_type_desc',
    'event_type_fin': 'fin_type_desc',})[colorder]
assert data.event_no.nunique() == data.shape[0]

Review data¶

Counts¶

Overall¶

• There are 394,199 gunfire-related 911 calls and ShotSpotter alerts prepared for this analysis.
• The data cover a time period between 2021-01-01 and 2024-11-04.

Source type¶

• Frequency table:

• Summary: Of the 394,199 emergency events included in the analysis,
  • 237,302 or 60.2% were generated by a 911 call, and
  • 156,897 or 39.8% were generated by a ShotSpotter alert.

5 most frequently reported disposition values¶

Presented are:

• the initial event type as reported by OEMC and CPD (init_type),
• the description of the initial type as found in the data (init_type_desc), and
• the type of source which reported the event (source_type).

Disposition¶

In the info page included with the data, the CPD FOIA officer informed us that they had internally identified emergency events from both sources, 911 callers and ShotSpotter, that referred to the same underlying gunfire event, and that the disposition field was only included in the responsive records when ShotSpotter was the first to report. Source

• Of the 394,199 emergency events included in the analysis, 123,216 or 31.3% have a reported disposition.

5 most frequently reported disposition values¶

• Of the 123,216 emergency events about potential gunfire identified by CPD as first reported by ShotSpotter, 104,586 or 84.9% are labeled as a "Miscellaneous Incident."